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Sir: 

Before examination of the above-identified application, 
please amend the application as follows: 



IN THE SPECIFICATION 



After the Title of the invention, insert the following: 
--CROSS-REFERENCE TO RELATED APPLICATION 

This is a divisional application of prior application No. 
08/437,563, of which this application claims priority under 35 
U.S.C. §120.-- 



IN THE CLAIMS 



Please CANCEL claims 2-10, without prejudice. 



REMARKS 



The present divisional application is filed to continue 
prosecution of non-elected claim 1 as was originally- filed in the 
parent application No. 08/437,563. In accordance with the 
foregoing, amendments to the specification are presented, 
corresponding to amendments entered in the allowed, parent 
application. 

It is respectfully requested that this Preliminary Amendment 
be entered in the above -referenced application. 

If any further fees are required in connection with the 
filing of this Preliminary Amendment, please charge same to our 
Deposit Account No. 19-3935. 



Respectfully submitted, 
STAAS & HALSEY 
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Washington, D.C. 20006 
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PATENT APPLICATION 



APPARATUS FOR MANIPULATING AN OBJECT 
DISPLAYED ON A DISPLAY DEVICE BY USING A TOUCH SCREEN 

BACKGROUND OF THE INVENTION 
Field of the Invention 

The present invention relates to an apparatus for 
manipulating an object displayed on a display device by using a 
touch screen. 

As use of computer systems for data processing has become 
widespread in recent years, more and more users are being 
required to handle same, i.e., to input data and converse with 
data processors such as work stations and personal computers. A 
vast variety of application programs are available for recent 
data processors and even a complicated application can be 
processed by using such application programs in combination. 
However, it is a great problem that such data processors are very 
difficult to handle, especially to manipulate an object displayed 
on a display device, for those who have less than a substantial 
knowledge of computers. 

Therefore, an apparatus for manipulating an object displayed 
on a display device, which is easily handled even for a person 
who has no special knowledge of computers, is in great demand. 

Description of the Related Art 

Fig. 1 illustrates a computer system with a conventional 
user interface. 

A computer system with a conventional user interface 
consists mainly of a central processing unit (CPU) 4, a main 
memory 5, a keyboard/mouse 2, a frame memory 6 and a hard disk 
interface 71, which are interconnected via a system bus, and also 
a hard disk 7 and a display unit 3, which are connected to system 
bus via the hard disk interface and the frame memory 6, 
respectively. The main memory 5 stores a system control program 
and application programs which handle graphics processing, and 
provides a work area for use by the programs. The CPU 4 performs 



display operations under control of the programs. The hard disk 
7 stores a data file for graphics to be displayed on the display 
unit 3. The frame memory 6 stores a frame of picture (or object) 
data to be displayed on the display unit 3. 

To manipulate an object displayed on a display unit 3 in the 
above system, an operator is required to input a command, for 
manipulating the object by using a keyboard/mouse 2, or to select 
an icon (a symbolic representation of a computer function) , 
displayed on a display unit 3 by using the keyboard /mouse 2, in 
order to command a desired function. However, it is troublesome 
and annoying to use a keyboard/mouse and icons and a person with 
less than a substantial knowledge of computers tends to be 
reluctant even to touch a keyboard/mouse. 

Therefore, it is a great problem that such data processors 
are very difficult to handle for those who have less than a 
substantial knowledge of computers. 

SUMMARY OF THE INVENTION 
It is an object of the present invention to provide an 
apparatus which can easily manipulate an object displayed on a 
display unit. 

It is another object of the present invention to provide a 
user interface with which a user can easily manipulate an object 
displayed on a display unit. 

To achieve the above and other objects, the present 
invention provides a touch screen, a plurality of data files, 
display information storage means and display control means. 

In the apparatus for manipulating an object displayed on a 
display device, the touch screen, which is a transparent panel 
and is mounted on the display surface, or screen, of a display 
device and is sensitive to the touch, e.g., of a finger of a 
human operator, outputs touch screen information representing the 
motion of the body. The plurality of data files store object 
data for displaying the object in different states. The display 
information storage means stores object information including at 



least an object type which specifies the shape and physical 
properties of the object, display position information which 
specifies a position where the object is displayed on the display 
device, file information which specifies the size and location of 
5 a part of the object data stored in one of said plurality of data 
files, and a file name which specifies one of said plurality of 
data files. The display control means recognizes a manipulation 
to be conducted on the object, based on the touch screen 
information from the touch screen and on the object information 
10 included in the display information storage means, and displays 

the object on the display device in accordance with the aforesaid 
recognition. 

BRIEF DESCRIPTION OF THE DRAWINGS 
Fig. 1 illustrates a computer system with a conventional 
15 3 user interface; 

Vi Fig. 2 is a block diagram of the configuration of a touch 

screen-equipped workstation, in which the present invention is 
implemented; 

Fig. 3 is a schematic diagram illustrating the principle of 
20 0 the present invention; 

Fig. 4(a) is a display information table; 
i3 Fig. 4(b) illustrates the format of touch-screen 

information; 

Fig. 5 is a flowchart illustrating a pick manipulation; 
25 Figs. 6(a) and 6(b) are related and interconnected diagrams 

illustrating and Fig. 6(c) is a table explaining a pick 
manipulation ; 

Fig. 7(a) is a diagram illustrating and Fig. 7(b) is a table 
explaining a scroll manipulation; 
30 Figs. 8(a) and 8(b) are related diagrams illustrating and 

Fig. 8(c) is a table explaining a push manipulation; 

Figs. 9(a) and 9(b) are related diagrams illustrating and 
Fig. 9(c) is a table explaining a flip manipulation; 
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Fig. 10(a) is a diagram illustrating and Fig. 10(b) is a 
table explaining a roll manipulation; 

Fig. 11(a) is a diagram illustrating and Fig. 11(b) is a 
table explaining a distort-restore manipulation. 

Throughout the above-mentioned drawings, identical reference 
numerals are used to designate the same or similar component 
parts . 

DESCRIPTION OF THE PREFERRED EMBODIMENTS 
Fig. 2 is a block diagram of the configuration of a touch 
screen-equipped workstation, in which the present invention is 
implemented. 

In addition to the conventional system shown in Fig. 1, the 
system of Fig. 2, for implementing the present invention, 
comprises an input-output (abbreviated to I/O) port 8, a touch 
screen controller 15 and a touch screen unit 1 with a touch 
screen 11. The touch screen controller 15, connected to the 
input-output port 8 through an RS-232C interface, controls the 
touch screen unit 1. The touch screen unit 1, which is 
sensitive, at a position (defined by X-Y coordinates) at which 
the touch screen 11 is touched and particularly to the pressure 
applied thereon when so touched, acts as a user interface that 
allows a user to send signals to a computer by touching that area 
thereon with an element, such as the finger, ... etc. 

Fig. 3 is a schematic diagram illustrating the principle of 
the present invention. 

For easy understanding of the principle, the input-output 
port 8, touch screen controller 15 and touch screen unit 1 shown 
in Fig. 2 are represented in Fig. 3, in the composite, by the 
touch screen unit 1 and the frame memory 6 and display unit 3 
and, as before-noted, the touch screen 11 actually is a 
transparent panel which is mounted on the display surface, or 
screen, of the display unit 3 but is shown separately therefrom 
in Fig. 3 of Fig. 2 are represented in Fig. 3, in the composite, 
by the display unit 3. The hard disk interface 71 and hard disk 



7 of Fig. 2 are represented in Fig. 3, in the composite, by the 
hard disk 7. A system controller 50, a touch discriminator 51, a 
display controller 52, which are programs stored in the memory 5 
and executed by the CPU 4 (or may be constructed by hardware) , 
and a display information table IT, stored in the main memory 5, 
control the display operations performed by the present 
invention. 

Fig. 4(a) shows a display information table. Fig. 4(b) 
shows touch screen information. 

The display information table IT, which is provided in the 
main memory 5 and corresponds to objects, includes object type 
information, display position information, file information, 
normal-display file name and special-state file name. The object 
type defines the type, including the shape, properties, 
circumstances, etc., of the object. The display position 
information defines the size of the object (width "W" and height 
"H") , and the position (top-left coordinates X,Y) and the angle 
at which the object is displayed on the display unit 1. The file 
information, which is used for an object which is so large in 
size that it requires scrolling to view the whole object, defines 
the size (width W, height H) of the whole object relative to the 
display screen size, and also the position (top-left coordinates 
X, Y) of the object portion being displayed on the display 
device, relative to the whole object whose data is stored in the 
normal display file. The normal-display file name specifies a 
display data file where object data for displaying a normal state 
of the object is stored. The special-state file name specifies a 
display data file where object data for displaying a special data 
(e.g., turn-over indication of display color, used for 
selectively displaying an intermediate process step, or stage, in 
manipulating the object) of the object is stored. The special 
state can be displayed selectively for each manipulation. 

Touch-screen information 21, which is sent from the touch 
screen unit 1, includes a touch position (X-Y coordinates) where 
the touch screen 11 is touched and the pressure applied thereon. 



The touch-screen information 21 may include two sets of X-Y 
coordinates shown in Fig* 4(b) depending on the type of touch, 
one set for a thumb and another for an index finger when the two 
fingers pick the object on its opposite sides, for example. 
(1) The touch discriminator 51, based on the touch screen 
information 21 from the touch screen unit 1, discriminates the 
type of touch which the operator's finger has on the touch screen 
11, that is, a touch type including, e.g., a "continuous touch 
start" and "continuous touch end" as explained later. The touch 
discriminator 51 sends, to the system controller 50, the result 
of the touch discrimination performed thereby as a touch report 
3R, which includes touch type and, according to the touch-screen 
information 21, one or two sets of touch coordinates. 

Based on the touch report 3R from the touch discriminator 51 
and the display information table IT, the system controller 50 
determines the type of manipulation which was conducted by the 
operator and, according to the determination, updates the display 
information table IT. Then, the system controller 50 sends, to 
the display controller 52, a display update request 4Q including 
"display update data" which reflects the updated contents of the 
display information table IT including display position 
information, filed information and normal display file name and 
special state file name. 

On receipt of the display update request 4Q from the system 
controller 50, the display controller 52 reads a display data 
file (including object data), specified by the file name, from 
the hard disk 7 and stores the data into the main memory 5. The 
display controller 52 then updates the object data in accordance 
with the display update data from the system controller 50 and 
loads the thus-updated object data into the frame memory 6 (Fig. 
2) thereby to display the object on the display unit 3 (Fig. 2), 
as manipulated by the operator on the touch screen unit 1. 

Thus, the system of the present invention determines a 
manipulation to be conducted on the displayed object, based on 
the touch screen information 21 which results from an operator's 
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touching the touch screen 11 and the display information table IT 
(see. Fig. 7(b)) which defines the object's shape, physical 
properties, display position, etc. The system then displays the 
object, according to the manipulation as thus determined and as 
5 intended by the operator, on the display unit 3. 

(2) Pick manipulation (see Figs. 5 and 6(a) to 6(c)). 

A pick manipulation is conducted in such a way that an 
object is picked up at a position on the display surface of the 
display unit 3 and placed at another position thereon. 
10 Fig. 5 is a flowchart illustrating a pick manipulation. 

Figs. 6(a) and 6(b) are related and interconnected diagrams 
illustrating, and Fig. 6(c) is a table explaining, a pick 
manipulation. 

:i A pick manipulation is carried out according to the 

15 U following steps (S1-S8) in Fig. 5: 

Z. (SI) The system controller 50 receives a touch report 3R 

including two sets of touch coordinates, from the touch 
\Z discriminator 51. 

(S2) The system controller 50 (Fig. 3) checks whether the 
20 object-finger relation is a pick manipulation, as shown in Fig. 
n 6(a), based on the touch report 3R and contents of the display 
W information table IT shown in Fig. 6(c). When the relation is 
not a pick manipulation, the system controller 50 checks the 
touch report 3R for other possible manipulations. 
25 (S3) When the relation is a pick manipulation, the system 

controller 50 sends a display update request 4Q including 
"display update data", commanding that the special-state file 
(turn-over indication) be displayed at the position specified by 
the display information table IT. 
30 (S4) The system controller 50 receives a touch report 3R. 

(S5) The system controller 50 determines whether the touch 
report 3R includes a "continuous touch end", which occurs when 
the finger-object relation is as shown in Fig. 6(b). When a 
"continuous touch end" is reported, the operation goes to step 
35 (S8). 



(56) Otherwise, the system controller 50 updates the display 
position information "coordinates (X, Y) " of the display 
information table IT so that the object is positioned between the 
two fingers of the operator, 

(57) The system controller 50 sends display update request 
4Q to the display controller 52, commanding that the 
special-state file be displayed according to the display 
information table IT, as updated, and returns to step (S4) • 

(58) When "continuous touch end" is reported by a touch 
report 3R, the system controller 50 sends a display update 
request 4Q to the display controller 52, commanding that the 
normal-display file be displayed at the position specified in the 
display information table IT. 

The following manipulations are carried out in the same way 
as descried in the above flowchart of the pick manipulation. 
(3) Scroll manipulation (see Figs. 7(a) and 7(b)). 

A scroll manipulation is conducted in such a way that an 
object, extending outside of the display surface of the display 
unit 3, is moved into and out of the display surface. 

Fig. 7(a) is a diagram illustrating, and Fig. 7(b) is a 
table explaining, a scroll manipulation. 

On determining that the operator's finger moves while 
touching the touch screen 11, based on the touch screen 
information 21 from the touch screen unit 1, the discriminator 51 
sends, to the system controller 50, a touch report 3R as 
previously discussed including, e.g., "continuous touch start" 
for the "touch type" and also, e.g., "coordinates (800, 800)" for 
the touch position. As another touch screen information 21 comes 
in, the discriminator 51 sends a touch report 3R including, e.g., 
"continuous touch in progress" and "coordinates (780, 800)" 
(i.e., the movement to the left by the finger as shown in Fig. 
7(a), and thus from X = 800 to X = 780 while at a fixed Y = 800). 
When the touch screen information 21 is not sent for more than 
100 milliseconds, for example, the discriminator 51 sends a touch 
report 3R including "continuous touch end" and, e.g., 
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"coordinates (700, 800)" (i.e., the final X,Y coordinate as of 
the "touch end") to the system controller 50. 

When a "continuous touch start" is reported and the "object 
type" is defined as "out-screen" in the display information table 
5 IT as shown in Fig. 7(b), the system controller 50 recognizes the 
manipulation as a scroll and the object as a large one extending 
beyond the display screen. Then, the system controller 50 
determines the speed at which the finger has moved from right to 
left, for example, based on a change in the X-coordinate in terms 

10 of data, between a touch report 3R and the following one. 

Depending on whether the finger has moved at a speed of more 
(high-speed) or less (normal -speed) than 20 dots, for example, 
the object display position on the display screen is scrolled 
n initially at corresponding intervals of 100 or 500 milliseconds, 

15 respectively. Then, the interval, at which the display update 
" request 4Q is sent to the display controller 52, is increased by 
a factor of 1.5 at each touch report 3R and, when the interval 
reaches 2 seconds, the scrolling is stopped. 

Practically, the display position area of the screen is so 

20'; controlled that it starts scrolling at the appropriate speed, as 
above-mentioned, after the operator's finger has moved a distance 
of 4 dots or more. That is, on recognizing that the finger has 
moved by that distance, the system controller 50 updates the file 
information "display position X" of the display information table 

25 IT so that the object is displayed, shifted to the left by 10 

dots, for example. The, it sends, to the display controller 52, 
a display update request including display position information, 
file information and normal display file name from the display 
information table IT, as updated. 

3 0 The display controller 52 reads from the hard disk 7 a 

display file specified by the normal display file name and loads 
it in the main memory 5. The display controller 52 then 
transfers only the part of the display file, as specified by the 
file information "display position X" of the display information 
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table IT, from the main memory 5 to the appropriate location of 
the frame memory 6. 

In the same way, the system controller 50 sends a display 
update request 4Q to the display controller 52 every time it 
5 receives a touch report 3R. 

When another "continuous touch" is reported before the 
scroll currently in progress comes to a stop, a new scroll can 
start from this point and at the first speed described above • 
(4) Scroll-stop manipulation (see Figs. 7(a) and 7(b)). 
10 Fig. 7(a) is a diagram illustrating, and Fig. 7(b) is a 

table explaining, a scroll manipulation. 

When a touch position given by a touch report 3R is the same 
as or up to approximately 5 dots apart from the position of the 
scrolling currently in progress, the system controller 50 doubles 
15 the frequency with which display update request 4Q are sent to 
the display controller 52, in order to put an end to the 
ry scrolling. 

■I (5) Push manipulation (see Figs. 8(a) to 8(c)). 

A push manipulation is conducted in such a way that an 
20 i object is pushed on the display surface of the display unit 3. 

Fig. 8(a) is a diagram illustrating, and Fig. 8(c) is a 
table explaining, a push manipulation. 

The system controller 50 determines the type of a 
manipulation, based on the touch report 3R and contents of the 
25 display information table IT shown in Fig. 8(c). When the 

manipulation is a push manipulation as shown in Fig. 8(a), the 
system controller 50 sends, to the display controller 52, a 
display update request 4Q including display position information, 
file information and normal display file name so that the object 
30 is displayed close to the finger position reported by the touch 
report 3R. The above display operation is repeated until a 
"continuous touch end" is reported by a touch report 3R. 
(6) Push-while-rotate manipulation (see Figs. 8(b) and 8(c)). 

A push-while-rotate manipulation is conducted in such a way 
35 that an object is pushed at a position off its center (or the 
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center of gravity shown as X in Fig^ 8(b)) and it moves rotating 
on the display surface of the display unit 3. 

The system controller 50 determines the type of a 
manipulation, based on the touch report 3R and contents of the 
display information table IT shown in Fig. 8(c). When the 
manipulation is a push-while-rotate manipulation as shown in Fig. 
8(b), the system controller 50 sends, to the display controller 
52, display update requests 4Q with the angle of rotation 
increasing by 2 degrees, i.e., while increasing the angle in the 
display information table IT shown in Fig. 8(c). 

The display controller 52 reads the display file from the 
hard disk and loads the data in the main memory 5, rotates the 
object by the specified angle and with the left-top coordinates 
(X, Y) as a rotational center, as specified by the display update 
request 4Q, and transfers the data, with the object rotated, from 
the main memory 5 to the frame memory 6- 
(7) Flip manipulation (see Figs. 9(a) to 9(c)). 

A flip manipulation is conducted in such a way that an 
operator's finger flips an object or touches (i.e., impacts) the 
object from a remote position at a high speed on the touch screen 
11 with a corresponding result on the display surface of the 
display unit 3. 

Fig. 9(a) is a diagram illustrating, and Fig. 9(c) is a 
table explaining, a flip manipulation. 

vnien a touch report 3R is input from the touch discriminator 
51, the system controller 50 discriminates the type of the 
manipulation, based on the touch report 3R and the contents of 
the display information table IT shown in Fig. 9(c). When the 
manipulation is a flip manipulation as shown in Fig. 9(a), the 
system controller 50 obtains the finger speed based on the touch 
report 3R and also obtain the object speed (i.e., the respective 
intervals at which display update requests 4Q are sent to the 
display controller 52) , in the same way as described in item (3) . 
The system controller 50 sends display update requests 4Q to the 
display controller 52, while updating the display position 
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information left-top coordinates (X, Y) of the display 
information table IT so that the object moves in the direction 
the finger moves. The system controller 50 stops moving the 
object when the above-mentioned interval reaches 2 seconds. 
(8) Flip-under-gravity manipulation (see Figs. 9(b) and 9(c)). 

A flip-under-gravity manipulation is conducted in such a way 
that an object, which is subjected to gravity, is flipped by a 
finger on the touch screen 11 and with a corresponding result on 
the display surface of the display unit 3. 

Fig. 9(a) is a diagram illustrating, and Fig. 9(c) is a 
table explaining, a flip manipulation. 

When the finger manipulation is a flip as in the above item 

(8) and the display information table IT defines the object type 
as "gravity" meaning that the object is subjected to gravity, for 
example, the object moves while receiving a combination of the 
forces of inertia and gravity, as shown in Fig. 9(b). Therefore, 
the system controller 50 sends display update requests 4Q to the 
display controller 52, while updating the display position 
information left-top coordinates (X, Y) by adding a value to the 
Y-coordinate of the display information table IT. The value is 
represented by 2 to the Nth power (N: the number of display 
update requests 4Q which are sent) . In this case, too, the 
system controller 50 stops moving the object when the 
above-mentioned interval reaches 2 seconds. 

(9) Roll manipulation (see Figs. 10(a) and 10(b)). 

A roll manipulation is conducted in such a way that a 
reliable object is rolled by the operator's finger on the touch 
screen 11 and with a corresponding result on the display surface 
of the display unit 3. 

Fig. 10(a) is a diagram illustrating, and Fig. 10(b) is a 
table explaining, a roll manipulation. 

When a touch report 3R is input from the touch discriminator 
51 and the display information table IT defines the object type 
as "reliable", meaning that the object is constructed such that 
it rolls when pushed by a finger, like a globe or a cylinder and 
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as shown in Fig, 10(a), the system controller 50 sends display 
update requests 4Q to the display controller 52, while updating 
the display position information left-top coordinates (X, Y) of 
the display information table IT so that the object moves a 
distance 10 percent behind the distance moves on the display 
screen and in the direction the finger moves. The system 
controller 50 sends display update request 4Q to the display 
controller 52 until "continuous touch end" is reported from the 
touch discriminator 51. 

(10) Distort-restore manipulation (see Figs. 11(a) and 11(b)). 

A distort-restore manipulation is conducted in such a way 
that an elastic object is pressed by a finger on the display 
surface of the display unit 3. 

Fig. 11(a) is a diagram illustrating, and Fig. 11(b) is a 
table explaining, a distort-restore manipulation. 

When a touch report 3R is input from the touch discriminator 
51 and the display information table IT defines the object type 
as "elastic", meaning that the object can be distorted and 
restored according to a pressure applied thereon by a finger and 
as shown in Fig. 11(a), the system controller 50 calculates an 
amount of distortion of the object based on the pressure reported 
by the touch report 3R. It stores in the display information 
table IT, a special-state file name specifying a corresponding 
one of the special-state files (for displaying a distorted state 
of the object in a turn-over indication) corresponding to the 
amount of distortion calculated. Then, the system controller 50 
sends a display update request 4Q to the display controller 52, 
commanding that the special-state file be displayed at the 
current display position. The above operation is repeated as 
necessary and when a "continuous touch end" is reported by a 
touch report 3R, the system controller 50 sends a display update 
request 4Q (with a normal display file name specified) to the 
display controller 52, commanding that a normal display file 
(normal indication) be displayed at the current display position. 
A plurality of special-state files are provided in the hard disk 
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7, corresponding to the amount of distortion of the object which 
results from the pressure applied on the touch screen 11. 

As is apparent from the above description, the present 
invention regards a display screen as a visual space. It defines 
conditions and physical properties of an object (e.g., weight, 
hardness, frictional resistance, center of gravity) in the 
display information table IT. It also receives touch screen 
information 21, indicating a finger-touched position and 
pressure, as input from a touch screen unit 1. Based on the 
touch screen information 21 and the display information table IT, 
the present invention determines a manipulation to be conducted 
on the displayed object, e.g, scrolling, picking, pushing, 
rolling, distorting the object on the display surface of the 
display unit 3. Thus, the present invention allows a user to 
manipulate an object displayed on a display device quite easily, 
even when the user has little knowledge of computers. 
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What is claimed is: 

1. An apparatus for manipulating an object displayed on a 
display device, comprising: 

a touch screen, which is provided on a display surface of 
the display device and is sensitive at least to a position 
thereon at which a body touches, for outputting touch screen 
information representing a motion of the body; 

a plurality of data files which store object data for 
displaying the object in different states; 

display information storage means for storing object 
information including at least 

an object type which specifies the shape and physical 
properties of the object, 

display position information which specifies a position 
where the object is displayed on the display device, 

file information which specifies the size and location 
of a part of the object data stored in one of said plurality 
of data files, and 

a file name which specifies one of said plurality of 
data files; and 

display control means, based on the touch screen information 
from said touch screen and on the object information included in 
said display information storage means, for recognizing a 
manipulation to be conducted on the object and for displaying the 
object on the display device in accordance with the recognition. 

2. An apparatus for manipulating an object displayed on a 
display device, according to claim 1, wherein 

based on the touch screen information from said touch screen 
representing that two bodies touch both sides of the object, move 
and stop with a distance in between, said display control means 
recognizes a pick manipulation and displays the object on the 
display device so that the object moves on the display surface of 
the display device from where the two bodies touch the both sides 
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of the object to where the two bodies stop with a distance in 
between. 

3. An apparatus for manipulating an object displayed on a 
display device, according to claim 1, wherein 

based on the touch screen information from said touch screen 
representing that a body touches the object and moves on said 
touch screen while keeping touched the object, and based on the 
object information specifying the object type as "out-screen" in 
said display information storage means, which means that the 
object is a large one extending beyond the display screen, said 
display control means recognizes a scroll manipulation and 
displays the object on the display device so that the object 
scrolls on the display surface of the display device. 

4. An apparatus for manipulating an object displayed on a 
display device, according to claim 3, wherein 

based on the touch screen information from said touch screen 
representing that the moving body stops, said display control 
means recognizes a scroll-stop manipulation and displays the 
object on the display device so that the scrolling object stops 
on the display surface of the display device. 

5. An apparatus for manipulating an object displayed on a 
display device, according to claim 1, wherein 

based on the touch screen information from said touch screen 
representing that a body touches the object at the center or the 
center of gravity, moves and stops on said touch screen while 
keeping touched the object, said display control means recognizes 
a push manipulation and displays the object on the display device 
so that the object moves on the display surface of the display 
device from where the body touches the object to where the body 
stops . 
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6. An apparatus for manipulating an object displayed on a 
display device, according to claim 1, wherein 

based on the touch screen information from said touch screen 
representing that a body touches the object at a position off the 
center or the center of gravity thereof, moves and stops on said 
touch screen while keeping touched the object, said display 
control means recognizes a push-while rotate manipulation and 
displays the object on the display device so that the object 
moves while rotating on the display surface of the display device 
from where the body touches the object to where the body stops. 

7. An apparatus for manipulating an object displayed on a 
display device, according to claim 1, wherein 

based on the touch screen information from said touch screen 
representing that a body touches the object from a position apart 
therefrom at a speed higher than a predetermined speed, said 
display control means recognizes a flip manipulation and displays 
the object on the display device so that the object moves a 
distance proportional to the speed with which the body touches 
the object and in the direction toward which the body touches the 
ob j ect . 

8. An apparatus for manipulating an object displayed on a 
display device, according to claim 1, wherein 

based on the touch screen information from said touch screen 
representing that a body touches the object from a position apart 
therefrom at a speed higher than a predetermined speed, and based 
on the object information specifying the object type as "gravity" 
in said display information storage means, which means that the 
object is subjected to a gravity, said display control means 
recognizes a f lip-under-gravity manipulation and displays the 
object on the display device so that the object moves a distance 
proportional to the speed with which the body touches the object 
and along a parabola which the object will describe when it 



17 



starts traveling in the direction the body touches while 
receiving the gravity downward, 

9. An apparatus for manipulating an object displayed on a 
display device, according to claim 1, wherein 

based on the touch screen information from said touch screen 
representing that a body touches the object, moves and stops on 
said touch screen while keeping touched the object, and based on 
the object information specifying the object type as "reliable" 
in said display information storage means, said display control 
means recognizes a roll manipulation and displays the object on 
the display device so that the object moves with positional 
relations between the object and the body varying from those at 
the beginning as the body moves. 

10. An apparatus for manipulating an object displayed on a 
display device, according to claim 1, 

wherein said touch screen is sensitive to an amount of 
pressure applied thereon, and 

wherein based on the touch screen information from said 
touch screen representing that a body touches the object with an 
amount of pressure, and based on the object information 
specifying the object type as "elastic" in said display 
information storage means, said display control means recognizes 
a distort-restore manipulation and displays the object on the 
display device so that the object varies in the degree of 
distortion and restoration according to the amount of pressure 
applied. 
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ABSTRACT OF THE DISCLOSURE 

Apparatus and a method of operation manipulates the 
display of an object on the display surface of a display 
device in accordance with sensing touching contact on a 
touch panel, superimposed on the display surface and 
simulating movement of the displayed object in accordance 
with the characteristics of the touching contact on the 
touch panel adjacent to the displayed object image. 
Further, object data defining the display of the object 
image for each of a plurality of different states of the 
object, display information specifying the shape and 
physical properties of the object and the current display 
position thereof and file information relating to the stored 
data are stored and accessed for interpreting the simulated 
manipulation of the object by the characteristics of the 
touching contact therewith, for correspondingly manipulating 
and displaying the thus manipulated object image* 
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